Building A Rendering System with MapReduce Framework ?
نویسندگان
چکیده
3D rendering is a kind of application which is not only computation intensive but also data intensive. 3D rendering can be easily parallelized by rendering different frames at the same time and the rendering work of each frame is independent from others. Parallel 3D rendering is typically I/O bound: many rendering programs spend lots of their execution time reading data from data server rather than performing useful computation. Furthermore the exceptions of the hardware and software are very common during the rendering work. To solve the problem mentioned above, we develop a rendering system based on the MapReduce software framework. Hadoop is an open source implementation of the MapReduce software framework and we use it to build our rendering system. Our rendering system tries to minimize data transmission to overcome the I/O bottleneck. The scheduler of hadoop is in charge of scheduling the rendering task of each frame.
منابع مشابه
Adaptive Dynamic Data Placement Algorithm for Hadoop in Heterogeneous Environments
Hadoop MapReduce framework is an important distributed processing model for large-scale data intensive applications. The current Hadoop and the existing Hadoop distributed file system’s rack-aware data placement strategy in MapReduce in the homogeneous Hadoop cluster assume that each node in a cluster has the same computing capacity and a same workload is assigned to each node. Default Hadoop d...
متن کاملEvaluating MapReduce System Performance: A Simulation Approach
Scale of data generated and processed is exploding in the Big Data era. The MapReduce system popularized by open-source Hadoop is a powerful tool for the exploding data problem, and is widely employed in many areas involving large scale of data. In many circumstances, hypothetical MapReduce systems must be evaluated, e.g. to provision a new MapReduce system to provide certain performance goal, ...
متن کاملBuilding a Java MapReduce Framework for Multi-core Architectures
MapReduce is a programming pattern that has been proved to be a simple abstraction on top of which can be built an efficient platform for largescale data processing in distributed environments, such as Google or Hadoop. With this pattern, application logic is expressed using sequential map and reduce functions. Thus, a runtime system can exploit the lack of side effects (pure functions) in thes...
متن کاملA Fully-Protected Large-Scale Email System Built on Map-Reduce Framework
Running an email system with full protection from spam and viruses has always been a pain for any system administrator. The problem becomes more severe for those who are responsible for large number of mail-boxes with huge amount of data in a large-scale email system. By using MapReduce framework, which is designed for distributed processing of large data sets on clusters of computers, this pap...
متن کاملA Hybrid Framework for Building an Efficient Incremental Intrusion Detection System
In this paper, a boosting-based incremental hybrid intrusion detection system is introduced. This system combines incremental misuse detection and incremental anomaly detection. We use boosting ensemble of weak classifiers to implement misuse intrusion detection system. It can identify new classes types of intrusions that do not exist in the training dataset for incremental misuse detection. As...
متن کامل